Session 6: ATIS Site Reports and General Discussion

Many of the evening session papers served to describe local implementations of ATIS domain systems and to provide local "glass-box" diagnostic evaluations to complement the NIST "black box" scoring. In some cases, these had the appearance of "nit-picking". However, since our collective intent was to develop and implement an evaluation procedure, the criticisms were generally constructive, and each site has much to contribute toward improved implementations of SLS performance evaluation procedures. Many of these observations underscore the complexity of developing standard test procedures in a new and challenging arena: spoken language systems.